Papers by Nimet Beyza Bozdag
Prior Beliefs Prejudice LLM-as-Judge: Evidence from Persuasion Evaluation (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models are increasingly used as judges to evaluate text quality, content and assess arguments. |
| Approach: | They propose to exploit belief-conditioned rating inflation by using persuasion-based probing to examine persuasive arguments. |
| Outcome: | The proposed model fails to evaluate persuasive arguments based on belief alignment . the model fails in three of the three tasks, with belief-conditioned rating inflation accounting for 88% of cases. |
DialDefer: A Framework for Detecting and Mitigating LLM Dialogic Deference (2026.acl-long)
Copied to clipboard
Parisa Rabbani, Priyam Sahoo, Ruben Mathew, Aishee Mondal, Harshita Ketharaman, Nimet Beyza Bozdag, Dilek Hakkani-Tür
| Challenge: | a single model can shift toward disagreement (skepticism) on graduate-level science and toward agreement (deference) on social judgment. |
| Approach: | They propose a framework to detect and mitigat framing-induced judgment shifts . they propose 'DialDefer' framework to help model disagreements and disagreements based on attribution . |
| Outcome: | The proposed framework detects and mitigates dialogic deference shifts in LLMs . human-vs-LLM attribution drives the largest shifts (17.7 pp swing) |